# Device Applications

MobileLLM-600M is an autoregressive language model developed by Meta, employing an optimized Transformer architecture specifically designed for resource-constrained device applications. This model incorporates key technologies such as the SwiGLU activation function, a deep and thin architecture, shared embeddings, and grouped query attention. MobileLLM-600M has shown a significant performance increase in zero-shot common sense reasoning tasks, achieving accuracy improvements of 2.7% and 4.3% compared to previous state-of-the-art models with 125M and 350M parameters, respectively. The design philosophy behind this model can be scaled to larger models, such as MobileLLM-1B and 1.5B, both of which have achieved state-of-the-art results.

MobileLLM-350M is an autoregressive language model developed by Meta, utilizing an optimized Transformer architecture tailored for device-side applications to meet the needs of resource-constrained environments. The model integrates key technologies such as SwiGLU activation function, deep thin architecture, embedding sharing, and grouped query attention, resulting in significant accuracy improvements in zero-shot commonsense reasoning tasks. MobileLLM-350M offers performance comparable to larger models while maintaining a small model size, making it an ideal choice for natural language processing applications on devices.